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ABSTRACT 

The maximum entropy method has been used to reconstruct images in a wide range of astro- 
nomical fields, but in its traditional form it is restricted to the reconstruction of strictly positive 
distributions. We present an extension of the standard method to include distributions that can 
take both positive and negative values. The method may therefore be applied to a much wider 
range of astronomical reconstruction problems. In particular, we derive the form of the en- 
tropy for positive/negative distributions and use direct counting arguments to find the form 
of the entropic prior. We also derive the measure on the space of positive/negative distribu- 
tions, which allows the definition of probability integrals and hence the proper quantification 
of errors. 
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1 INTRODUCTION 

The maximum entropy method (MEM) has been applied to numer- 
ous image reconstruction problems in astronomy and various other 
fields (for reviews see Gull & Skilling (1984), Titterington (1985) 
and Narayan & Nityananda (1986)). In its standard form, however, 
it is only applicable to distributions that are stricly positive. For 
most problems in astronomy, one is concerned with the reconstruc- 
tion of the intensity of an image from some convolved and noisy 
data, and so the restriction of positivity is not a problem. Neverthe- 
less, more recently, the astronomical community has become in- 
terested in the use of Bayesian methods to reconstruct images of 
the temperature fluctuations in the cosmic microwave background. 
In order to avoid the introduction of any bias into the reconstruc- 
tion of such an image, it is desirable to reconstruct the temperature 
fluctuations about some mean level, rather than introducing some 
arbitrary offset to ensure the positivity. Clearly such a problem re- 
quires the reconstruction to take both positive and negative values 
about the mean level, and this has lead to the consideration of how 
to apply MEM to such problems (Maisinger, Hobson & Lasenby 
1997; Hobson et al. 1997; lones et al. 1997). 

The theoretical foundations of MEM, when applied to strictly 
positive distributions, are discussed in some detail by Skilling 
(1988, 1989). From very general notions of subset indepen- 
dence, coordinate independence, system independence and scaling, 
Skilling (1988) shows that the assignment of a positive distribution 
f(x), given some definitive but incomplete constraints on it, must 
be made by maximising over / (subject to the imposed constraints) 
the entropy functional S[f,m] given by 



S[f,m}= I ' f{x)-m{x)-f{x)\n 



fix) 



m(x) 



dx. 



(1) 



In this expression m(x) is the Lebesgue measure associated with x, 
which must be given before the integral can be defined. In prac- 



tice, however, m(x) is more usually interpreted as the model to 
which f(x) defaults in the absence of any constraints. Indeed it 
is easily shown that S[f, in] possesses a single global maximum 
at f(x) = mix). It is also common in practice that the distribu- 
tion f(x) is digitised on to N equivalent cells to form the vector 
f = (/i ,/2, • ■ • and a similar digitisation is also performed for 
the model m(x). In this case, the integral in ^ is simply replaced 
by a sum over the cells, so that 



S[f,m] 



N 



fi-m -/,ln 



Si 



(2) 



The arguments leading to (^) do not, however, address the re- 
liability of this assignment of the distribution f . In other words, we 
are not able to quantify how much better is this assignment of f as 
compared to any other distribution permitted by the imposed con- 
straints. Moreover, experimental data are usually noisy and so do 
not constitute testable constraints on the distribution f , but instead 
define a likelihood function Pr(data|f). Thus, in order to apply a 
proper Bayesian analysis, it is necessary to derive the full entropic 
prior probability distribution Pr(f|m). Following Skilling (1989), 
this leads us to consider probability integrals over domains V in the 
space of distributions. Assuming f to be digitised on to N cells, as 
described above, we write 



Pr(f eV\m) 



d w fM(f)Pr(f|m) 



where M(f ) is the measure on the space of positive distributions. 
In order that we identify the most probable distribution with that 
assigned by maximising the entropy, we must have 



Pr(f eV|m) = — - — r / d w fM(f)*(aS[f.m]). 

Z s (a,m) Jv 



(3) 



where <I> is a monotonic function of the dimensionless argument 
aS[f , m]. Since S[f , m] has dimensions of integrated f, then a is also 
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dimensional and is not an absolute constant. The factor Zs(a, m) is 
required to ensure that Pr(f |m) is properly normalised and is given 
by 



can still consider the difference m 



Z s (a.m) = I d A 'fM(f)*(a5[f,m]). 



In order to find the function <t> it is necessary to consider a spe- 
cific distribution f that obeys the general notions of subset indepen- 
dence etc. mentioned above, but for which the integral Pr(f 6 V|m) 
can be calculated directly. The traditional example used (Frieden 
1972, Gull & Daniell 1979, Jaynes 1986) is that of a distribution 
f created by a 'team of monkeys' throwing balls randomly at the 
cells i = 1,2,... ,N. As shown by Skilling (1989), this leads to the 
identifications 



<J>(oc5[f,m]) 
Z s (a,m) 



exp(oc5[f,m]), 



as well as yielding an expression for the measure on the space of 
positive distributions which is given by 

N 1/9 

A knowledge of the measure is important, since it enables us to de- 
fine probability integrals over domains in the space of distributions. 

In this paper, we discuss the extension of MEM to distribu- 
tions that can take both positive and negative values. The resulting 
formalism is clearly applicable to the reconstruction of temperature 
fluctuations in the microwave background, as discussed above, but 
may also be applied to a much wider range of image reconstruction 
problems. In particular, we present derivations of the general form 
of the entropy for positive/negative distributions and calculate the 
corresponding entropic prior probability distribution. 



2 THE ENTROPY OF POSITIVE/NEGATIVE 
DISTRIBUTIONS 

It is clear from the logarithmic term in the entropy functional (^) 
that negative values are not permitted in the (digitised) distribution 
f . Nevertheless, the problem of assigning distributions that can take 
both positive and negative values has been briefly considered by 
Gull & Skilling (1990) for certain special cases. In this section, 
we extend their method to find the form of the entropy for general 
positive/negative distributions. 

The central idea is to express the general (positive/negative) 
distribution h as the difference of two strictly positive distributions 
f and g, i.e. 

h = f g. 

If m y and rrig are separate models for f and g respectively, then the 
total entropy can be written as 

fi 



5[h, my, m g ] = £ { fi ~ ( m f)i ~ fi ln 

1=1 I. 

N r 

+ £j«i-K)i-ah 



(m f )i 
Si 



(4) 



Our aim is to derive a form for S[h,my,rrig] in which only h ap- 
pears, rather than the two positive distributions f and g. We cannot, 
however, hope to replace the models my and rrij, by a single pos- 
itive/negative model m;, (say), since such a distribution could no 
longer be considered as an integration measure. Nevertheless, we 



trig as the model for the dis- 



tribution h. 

We may remove the explicit dependence of 5[h,my,m f ] on 
the positive distributions f and g by applying continuity constraints 
on the entropy functional. Since h = f g, we have 



as 

dfi 
35 
9^ 



95 dh k 



N 

£ 'a** a/* 

* 95 dh k 
dh k dgi 



95 



35 

Mi' 



from which we obtain the constraint 
35 35 

Wi + Wi 

It is straightforward to show that 



= 0. 



(5) 



35 

Wi 



-In 



fi 



(«/) 



and a similiar expression exists for 35/3g,-. Hence (^) becomes 



ln 



./;■ 



n f )i 



-ln 



gi 



= 0, 



and thus we obtain the constraint ffgj = (my);(m s );. Using this 
constraint to substitute separately for fj and gi in the relation 
h, = fi — gi, and remembering that fi, gi, (my); and (m g )i are all 
required to be strictly positive, we obtain 



liVi + h) 



(6) 
(7) 



gi = 2 Wi -hi)- 

where \|/,- = [hf + 4(/n/-),(m^),] '/ 2 . Finally, substituting the rela- 
tionships (^) and (Q) into the expression for the entropy given in 
(Q), after a little algebra we find the form of the entropy for the 
positive/negative distribution h is given by 



5[h,m/,m ? ] = £ -j - (m/); - (m g ); -/i,Tn 



2(m/)i 



(8) 



3 THE ENTROPIC PRIOR FOR POSITIVE/NEGATIVE 
DISTRIBUTIONS 

As discussed in Section [l| its still remains to quantify the assign- 
ment of the positive/negative distribution h, by deriving the en- 
tropic prior probability Pr(h my , m s ). We may calculate this prior 
by again considering the specific case in which h is created us- 
ing the traditional team of monkeys, but now throwing two types 
of 'balls', for example red ones and blue ones (each of quantum 
size q) randomly into the cells, with Poisson expectations A.,- and 
fjj respectively. The distribution h is then found by considering the 
number of red balls r,- minus the number of blue balls /?, in each cell. 
In particular, we define /,■ = r/q and g, = b/q, so that hj = mq where 
iij = rj — bf. We also define the models for the positive and negative 
portions of the distribution h in a similar way as (my)/ = Xiq and 
{m g )i = m- 

We may now consider the space of h to be constructed from 
microcells of volume q N , each one associated with one lattice point 
of integers n = [n\ ,ri2, ■ ■ ■ ,«at), where each integer may be positive 
or negative (or zero). Since each cell is independent of the others, 
the probability of obtaining a given set of occupation numbers is 
given by the product of the probability distributions for the occupa- 
tion number in each cell separately, i.e. 
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Pr(n|A,/*) = nPr(f! i |A < , rt ). 
1=1 

We see that in order to proceed further, we must consider the 
probability distribution of the difference of two independent Pois- 
son variates. This distribution is derived in the Appendix and is 

Pi(n i \X i , fli ) = exp[-(X i + tli )](^\ ' /„ j (2V^rt), 

where I n (x) is the modified Bessel function of order n. Thus, as we 
let the quantum size q — » 0, the probability that the distribution h 
lies in some domain V of distribution space is given by 

Pr(h £V\m f ,m g ) = 



X 



rl^h N /A.-W 2 , 

_nexp[-(X,-+ W )] - 4,(2^). (9) 



!=1 



In the limit q — » 0, the integers »,■ will in general become very 
large, and so we are led to consider the uniform asymptotic ex- 
pansion of a modified Bessel functions of large order, which reads 
(Abramowitz & Stegun 1972) 



1 



exp(;?p) 



In{nX) ~ (27^)1/2(1+^)1/4' 
where the variable P is given by 
p = (l+x2)V2 + ln 



(10) 



.1 + 0+x 2 ) 1 / 2 

Setting x = 2^/X/j/n in ( ^o[ ) and ( |ll| ) and rearranging, we obtain 



(11) 



(27t^)) 1 / 2 



2(^)'/ 2 



n + <|> 



where we have defined the quantity if 
this result into (^), we find 

* r d w h 

Pr(h e V m/.% = / ,, ; z — -r-zr 

where the function ^(0, A, fl) is given by 



exp(<|>). 

) = (« 2 + 4A, J u) 1 < /2 . Substituting 
exp£(0,A, M )], (12) 



£(0,A,M) = exp 



E(<l>i-^-«; 



= exp 



»=l 



i=\ 



Hi) 

N 



2(W /2 



exp 



-E"' ln 



/=l 



«i+<|>i 

<t>i+iy 
2%i 



Collecting together the exponential factors and writing the resulting 
expression in terms of h, my and m g , we obtain 



1 



9 ,=i 



2(m/),- 



(13) 



where \|/,- = = +4(my)j(m g ),] ' ' 2 , and on the RHS we recog- 
nise the entropy functional given in (^). Thus, substituting ( |l3| ) into 
(|l|), we find 



Pr(h 6 V m/,m 



?) /vllf,.::.. 



d^h exp(S(h,m/,m g )/^) 



(27tg) A '/ 2 



(14) 



We may now compare this result with our earlier expression 
fy. Aside from multiplicative constants which can be defined to 
be unity, we therefore identify the quantum size q of the 'balls' 
as q = 1 /a. As in the case of a strictly positive distribution, the 
monotonic function <5 of the entropy functional is identified as 



4>(aS(h, m /, m g )) = exp(aS(h, m f , m g )), 
which together with the normalisation constant 
Z s (a,m/,m g ) = (27t/a) w / 2 , 

defines the entropic prior on the space of positive/negative distribu- 
tions. Furthermore, from (|l4|), we also identify the measure on the 
space of positive/negative distributions as 

which enables us to define probability integrals over domains V in 
this space. 

As discussed by Skilling (1989), a natural interpretation of 
the measure is as the invariant volume (det G) 1 / 2 of a metric G 
defined on the space. Thus the natural metric for the space of posi- 
tive/negative distributions is simply 

G = aag(l/\|/ 1 ,l/ifc,-..,l/w)- 

Furthermore, it is straightforward to show from (||) that the ele- 
ments of the Hessian matrix of the entropy functional are given by 

9 2 5 



1 



dhidhj 



so that, as in the case of positive-only distributions, we may write 
the metric as 

G = -V h V h 5(h,m /) m g ). 

Thus the metric on the space of positive/negative distributions at 
any given point is equal to minus the Hessian matrix of the entropy 
functional at that point. 



4 CONCLUSIONS 

We have presented derivations of the form of the entropy for dis- 
tributions that can take both positive and negative values. We also 
use direct counting arguments to calculate the form of the entropic 
prior for such distributions. This enables the maximum entropy 
method to be applied directly to the reconstruction of general posi- 
tive/negative distributions. This method will therefore be of use in a 
much wider range of astronomical image reconstruction problems. 
The calculations presented here also yield the form of the measure 
on the space of positive/negative distributions, which enables the 
calculation of probability integrals over domains in this space, and 
therefore the proper quantification of errors on the reconstructed 
distribution. 
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APPENDIX A: THE DIFFERENCE OF TWO 
INDEPENDENT POISSON VARIATES 

Let us consider two independent random variables X and Y. If X is 
Poisson distributed with mean X and Y is Poisson distributed with 
mean /j then 



X" 

Pt(X = n) = — exp(-X), 



Pr(K = m) 



-exp(- J u). 



(Al) 
(A2) 



If we now consider the variable Z = X — Y, then its probability 
distribution is given by 



Pr(Z = n) 



£Pr(X = n + r)Pr(y = r) for n > 



(A3) 



£ Pi(X = r) Pr(r = |n| + r) for n < 0. 
Substituting the expressions Q and @ into @, we obtain 



cxp[-(X+n)] £ 



for n > 



r=0 



Pr(Z = n) = { ^ , |+r (A4) 

^[-(X+M)]f for^O. 

Provided n is a non-negative integer, however, the modified Bessel 
function of order n is given by 



i n ( x ) = r"j n (bc) = £ 



{x/2) 



n+2r 



r\(n + r)\ 



and so we may write the first sum in ( A4 ) as 
[hn '' X"l„(2^)(Xp)-"/ 2 

J 1 , 



/„(2 V / ^) 



Writing the second sum in ( A4 ) in a similar way, we thus obtain 

/ X \ "/ 2 

exp[-(X + fj)] - / B (2VV) forn > 
Pr(Z = «) = <( W 

exp[-{X+n)] (Q '"' /2 /|„|(20^) for n sC 0. 



If w is an integer, however, /_„ (x) = /„ (x) and so the final expres- 
sion for the probability distribution of Z is given for all integers n 
by 

Pr(Z = n) = exp[-(X+n)] (j) ' / n (2x/V). (A5) 

The probability distribution for difference of two independent Pois- 
son variates has also been considered by Stuart & Ord (1994), but 
only for the special case X = /j and with the result expressed only 
as an infinite series. 



A useful check that ( A5 ) is correctly normalised is provided by 



considering the generating function for modified Bessel functions, 
which reads 

In (x)t" = exp \x I t + 
n— — °o L V 

Substituting / = sJXjp and x = 2^JX)J into this expression, we find 



/IN"/ 2 
£ /„(2 V / ^) - 

1= — oo V^y 



exp 



exp(X + /j). 



Hence, substituting this result into (A5), we find that as required 

£ Pr(Z = n) =exp[-(X+fj)]exp(X+/j) = 1. 

n=—°° 

This paper has been produced using the Royal Astronomical Soci- 
ety /Blackwell Science LTgX style file. 
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